Consistency check script #114
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kvdblom
requested changes
Nov 19, 2025
| "textual description", | ||
| ] | ||
|
|
||
| UNIQUE_FIELDS = ["name", "reference", "implementation"] |
Collaborator
There was a problem hiding this comment.
This triggers an error when:
- reference is
''(empty), which is probably not what we want. Thinking about it: A reference can also introduce multiple problems, so does not need to be unique in any case? What do you think? - Similarly, I expect the same happens for the implementation field, which may also not need to be unique, because a single package may implement multiple problems/benchmarks. (I guess it might depend a bit on how specific we want the reference to the implementation to be, but it probably cannot always be specific enough to be unique.)
| import sys | ||
|
|
||
| # Define the required fields your YAML must have | ||
| REQUIRED_FIELDS = [ |
Collaborator
There was a problem hiding this comment.
We have the same list of fields in yaml_to_html.py (called default_columns). We should probably maintain it in a single place, and/or let one inherit the other?
|
|
||
|
|
||
| def check_fields(data): | ||
| if len(data) != len(REQUIRED_FIELDS): |
Collaborator
There was a problem hiding this comment.
I think this should test that there are at least this many fields. I would explicitly want people to add new fields for interesting properties we do not collect yet. Then:
- Properties not in the
REQUIRED_FIELDSshould then be checked against a (to be created)NOT_REQUIRED_FIELDSwhich would contain all other fields (might be empty for now). - A message should be returned listing the new fields (found in neither the required or not-required lists), to be verified by an OPL maintainer as actually new (not just a new name for an existing property), and then either added to the not required list or fixed (or requested as change on a PR) to have the correct existing name.
- Ideally all other checks are still done before such a list is returned, so we know everything else already passes the checks, and verifying new fields (or maybe other similar things) is all that needs to be done.
Collaborator
|
Something else that came to mind that I did not think about or try yesterday:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a preliminary script for checking consistency, along with a README detailing the envisioned github workflow.
Main question for review: Should I be checking for anything else? Is there anything that is too strict?
Closes #123